Character recognition device, mobile communication system, mobile terminal device, fixed station device, character recognition method and character recognition program

ABSTRACT

Words possibly included in a scene image shot by a mobile camera can be efficiently extracted using a word dictionary or a map database. Positional information acquiring means  101  measures a current position of the device to acquire positional information. Directional information acquiring means  102  detects a direction of the device to acquire directional information. Character recognizing means  104  determines a range of shooting of a scene image based on the current positional information and the directional information. The character recognizing means  104  extracts from a map database  103  information such as store names, building names, and place names associated with positions in the shooting range. Then the character recognizing means  104  conducts character recognition using word knowledge such as the extracted store names, building names, and place names.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.10/585,323, filed on Jul. 6, 2006, which is a national stage ofInternational Application No. PCT/JP2004/018059, filed on Dec. 3, 2004,claiming priority based on Japanese Patent Application No. 2004-002848,filed on Jan. 8, 2004, the contents of all of which are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present invention relates to a character recognition device torecognize characters included in an image, a mobile communicationsystem, a mobile terminal device, a fixed station device, a characterrecognition method, a character recognition program, an informationcommunication program of a mobile terminal device, and an informationextraction program of a fixed station device, and in particular, to acharacter recognition device to recognize characters included in animage, a mobile communication system, a mobile terminal device, a fixedstation device, a character recognition method, a character recognitionprogram, an information communication program, and an informationextraction program capable of reading character information such as aplace name, a road name, a store name, a building name, an advertisementstatement, and a person's name described in a road sign, a store'ssignboard, and a poster in an image of a scene shot by a mobile camera.

RELATED ART

There have been known character recognition devices to recognizecharacter information in an image of a scene shot by a video cameraand/or a digital camera. In these character recognition devices, wordknowledge (word information) regarding a character string of an objectfor recognition is generally used in character recognition processing toobtain high recognition accuracy even if there exists variation inlighting and character distortion.

For example, patent document 1 (Japanese Patent No. 3360030 (pages 4 to6, FIGS. 1 to 4)) describes a character recognition device that extractscharacter information of a signboard sponsor from a scene image. Thecharacter recognition device described in patent document 1 reads a nameof the signboard sponsor and a telephone number from the image. Thecharacter recognition device counts the number of matching characters inthe character string and in the words contained in a telephone directorydatabase or the like to calculate a recognition score. Thereafter, thecharacter recognition device assumes, as a final recognition result,signboard sponsor information for which the highest recognition score isobtained.

Also, patent document 2 (Japanese Patent Application Laid-Open No.2003-178067 (pages 14 and 15; FIG. 20)) describes an image processingsystem of portable terminal type to conduct character recognition of animage shot by a portable terminal. In the image processing system ofportable terminal type described in patent document 2, positionalinformation obtained from a Global Positioning System (GPS) or the likeis related to a word dictionary. Then, a server of the image processingsystem of portable terminal type conducts character recognition using aword dictionary associated with information of a current position of andevice (portable terminal). For example, when a user is in a restaurant,the server selects and uses a dedicated dictionary containing wordknowledge such as that of a menu of regional specialties, the wordknowledge being likely to be used in a restaurant.

DISCLOSURE OF THE INVENTION Problem to be Solved by the Invention

In the character recognition device described in document 1 or the imageprocessing system of portable terminal type described in patent document2, if unnecessarily many word knowledge pieces are employed, therelikely occurs that the recognition speed is lowered, and the recognitionaccuracy is reduced due to the increase in similar words or the like.Therefore, the character recognition speedup and accuracy improvementcannot be fully carried out.

Additionally, according to the character recognition device described inpatent document 1, the recognition accuracy is improved by comparisonwith words contained in a telephone directory database or the like.However, since it is required to check a large volume of word knowledgecontained in a telephone directory database or the like, there existspossibility that the speedup and the accuracy improvement cannot befully achieved in the character recognition processing.

Furthermore, in accordance with the image processing system of portableterminal type described in patent document 2, a dedicated worddictionary only including words that are likely to be included in aninput image is employed in the character recognition by using theinformation of the current position of the device, and hence thecharacter recognition speed can be increased to a certain extent and thecharacter recognition accuracy can be increased.

However, for a scene image (input image), since a camera shoots a sceneat a position apart from the camera about several meters to several tensof meters, if the dictionary is selected on the basis of only thepositional information from GPS, the character recognition is requiredto be conducted using the entire words knowledge (dedicated worddictionary) corresponding to a range within a circle having a radius ofseveral tens of meters where the center is the current position.Therefore, the words likely to be contained in the scene image shot by amobile camera cannot be fully efficiently extracted from a worddictionary or a map database, and hence there still exists room for thecharacter recognition speedup and accuracy improvement.

In addition, according to the image processing system of portableterminal type described in patent document 2, the word knowledgedirectly associated with the positional information is used for thecharacter recognition. However, word information of other kinds notdirectly associated with the positional information cannot be utilizedfor the character recognition.

It is therefore an object of the present invention to provide acharacter recognition device, a mobile communication system, a mobileterminal device, a fixed station device, a character recognition method,a character recognition program, an information communication program,and an information extraction program capable of efficiently extractingwords likely to be included in a scene image shot by a mobile camera, byuse of a word dictionary or a map database. Also, it is an object of thepresent invention to provide a character recognition device, a mobilecommunication system, a mobile terminal device, a fixed station device,a character recognition method, a character recognition program, aninformation communication program, and an information extraction programcapable of conducting character recognition by use of word informationof a kind not directly associated with positional information.

Means for Solving the Problem

A character recognition device in accordance with the present inventioncomprises imaging means for shooting an image; position measuring meansfor measuring a shooting position of an image to obtain shootingposition information indicating the shooting position; directiondetecting means for detecting a shooting direction of an image to obtainshooting direction information indicating the shooting direction;position corresponding information storage means for storing positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places;position corresponding information extracting means for determining,based on the shooting position information and the shooting directioninformation, a range of a shooting object of the imaging means, andextracting from the position corresponding information storage meansposition corresponding information associated with positions included inthe range; and character recognizing means for recognizing, using theposition corresponding information extracted by the positioncorresponding information extracting means, a character or a characterstring included in the image imaged by the imaging means. In thisregard, the position corresponding information is word information, forexample, the transportation network information, place name information,building information, and store information stored in the map database103.

Furthermore, the character recognition device may comprisenon-corresponding-to-position information storage means for storingtherein non-corresponding-to-position information that is words notassociated with positional information, andnon-corresponding-to-position information extracting means forextracting non-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means, wherein the characterrecognizing means recognizes a character or a character string includedin the image using the position corresponding information extracted bythe position corresponding information extracting means and thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means. In thisstructure, since non-geographical non-corresponding-to-positioninformation related to words extracted from the position correspondinginformation storage means can be read to be used for characterrecognition, recognition accuracy can be improved for character stringsor the like that cannot be directly associated with positionalinformation such as a person's name, an item name, or a date.

Moreover, it may be possible for the non-corresponding-to-positioninformation storage means to store, as non-corresponding-to-positioninformation, a set of words that are not associated with the positionalinformation and that include at least one word equal to positioncorresponding information; and the non-corresponding-to-positioninformation extracting means extracts non-corresponding-to-positioninformation from the non-corresponding-to-position information storagemeans using as a key the position corresponding information extracted bythe position corresponding information extracting means. In thisconfiguration, words included in non-corresponding-to-positioninformation can be easily extracted to be used for the characterrecognition processing.

Also, the mobile communication system in accordance with the presentinvention may be a mobile communication system in which a mobileterminal device and a fixed station device communicate via a wirelesstransmission path with each other, wherein the mobile terminal devicecomprises imaging means for shooting an image; position measuring meansfor measuring a shooting position of an image to obtain shootingposition information indicating the shooting position; directiondetecting means for detecting a shooting direction of an image to obtainshooting direction information indicating the shooting direction; andinformation transmitting means on mobile terminal side for transmittingthe shooting position information, the shooting direction information,and an image imaged by the imaging means via a communication network tothe fixed station device; and the fixed station device comprisesposition corresponding information storage means for storing positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places;position corresponding information extracting means for determining,based on the shooting position information and the shooting directioninformation that are received from the information transmitting means onmobile terminal side, a range of a shooting object of the imaging means,and extracting from the position corresponding information storage meansposition corresponding information associated with positions included inthe range; character recognizing means for recognizing, using theposition corresponding information extracted by the positioncorresponding information extracting means, a character or a characterstring included in the image which are received from the informationtransmitting means on mobile terminal side; and information transmittingmeans on fixed station side for transmitting information of a characteror a character string recognized by the character recognizing means viaa communication network to the mobile terminal device. In suchstructure, since the mobile terminal device need not include theposition corresponding information storage means and the characterrecognizing means, the configuration of the mobile terminal device canbe simplified.

Furthermore, it is possible for the fixed station device to comprisenon-corresponding-to-position information storage means for storingnon-corresponding-to-position information that is words not associatedwith positional information, and non-corresponding-to-positioninformation extracting means for extractingnon-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means, wherein the characterrecognizing means recognizes a character or a character string includedin the image using the position corresponding information extracted bythe position corresponding information extracting means and thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means. In suchconfiguration, the configuration of the mobile terminal device can besimplified and, character recognition accuracy can be increased for acharacter string or the like not directly related to positionalinformation.

Furthermore, the mobile terminal device may comprise informationreceiving means on mobile terminal side for receiving information of acharacter or a character string recognized by the character recognizingmeans via a communication network from the information transmittingmeans on fixed station side, and output means for outputting informationof the character or the character string. In such structure, the resultinformation of the character recognition can be supplied to the user ofthe mobile terminal by displaying the information on a display moduleand/or by producing sound and voice. In this connection, to outputinformation of a character or a character string is to display theinformation of a character or a character string on a display module orto output the information to a sound and voice output module.

Also, the mobile communication system may be a mobile communicationsystem in which a mobile terminal device and a fixed station devicecommunicate via a wireless transmission path with each other, whereinthe mobile terminal device comprises imaging means for shooting animage, position measuring means for measuring a shooting position of animage to obtain shooting position information indicating the shootingposition, direction detecting means for detecting a shooting directionof an image to obtain shooting direction information indicating theshooting direction, information transmitting means on mobile terminalside for transmitting the shooting position information and the shootingdirection information via a communication network to the fixed stationdevice, and character recognizing means for recognizing a character or acharacter string included in the image imaged by the imaging means; andthe fixed station device comprises position corresponding informationstorage means on fixed station side for storing position correspondinginformation that is words associated with respective positionalinformation indicating positions of respective places, positioncorresponding information extracting means on fixed station side fordetermining, according to the shooting position information and theshooting direction information that are received from the informationtransmitting means on mobile terminal side, a range of a shooting objectof the imaging means, and extracting from the position correspondinginformation storage means on fixed station side position correspondinginformation associated with positions included in the range;non-corresponding-to-position information storage means for storingnon-corresponding-to-position information that is words not associatedwith positional information, non-corresponding-to-position informationextracting means for extracting non-corresponding-to-positioninformation from the non-corresponding-to-position information storagemeans according to the position corresponding information extracted bythe position corresponding information extracting means on fixed stationside, and information transmitting means on fixed station side fortransmitting the non-corresponding-to-position information extracted bythe non-corresponding-to-position information extracting means via acommunication network to the mobile terminal device, wherein thecharacter recognizing means recognizes a character or a character stringincluded in the image using the non-corresponding-to-positioninformation received from the information transmitting means on fixedstation side. In such configuration, since the retrieval processing ofthe non-corresponding-to-position information is executed in the fixedstation, the processing load on the mobile terminal device is mitigatedand the mobile terminal device need not have high computationperformance. Additionally, since the database storing thenon-corresponding-to-position information is controlled on the fixedstation side in a unified manner, maintenance is facilitated andmaintainability can be improved.

Moreover, the mobile terminal device may comprise position correspondinginformation storage means on mobile terminal side for storing thereinposition corresponding information which is words associated withrespective positional information indicating positions of respectiveplaces and position corresponding information extracting means on mobileterminal side for determining, based on the shooting positioninformation and the shooting direction information, a range of ashooting object of the imaging means, and extracting from the positioncorresponding information storage means on mobile terminal side positioncorresponding information associated with positions included in therange, wherein the character recognizing means recognizes a character ora character string included in the image using thenon-corresponding-to-position information received from the informationtransmitting means on fixed station side and the position correspondinginformation extracted by the position corresponding informationextracting means on mobile terminal side. In such structure, bytransmitting the non-corresponding-to-position information, the volumeof data transmitted from the fixed station device to the mobile terminaldevice can be reduced.

Also, it is possible for the information transmitting means on fixedstation side to transmit the non-corresponding-to-position informationextracted by the non-corresponding-to-position information extractingmeans and the position corresponding information extracted by theposition corresponding information extracting means on fixed stationside via a communication network to the mobile terminal device and thecharacter recognizing means recognizes a character or a character stringincluded in the image using the non-corresponding-to-positioninformation and the position corresponding information that are receivedfrom the information transmitting means on fixed station side. In suchstructure, compared with a situation in which only thenon-corresponding-to-position information is transmitted, the processingload on the mobile terminal device can be further mitigated and themobile terminal device need no have high computation performance.

Moreover, it is possible for the non-corresponding-to-positioninformation storage means to store non-corresponding-to-positioninformation that is a set of words not associated with positionalinformation and includes at least one word equal to positioncorresponding information, and the non-corresponding-to-positioninformation extracting means extracts non-corresponding-to-positioninformation from the non-corresponding-to-position information storagemeans using as a key the position corresponding information extracted bythe position corresponding information extracting means on fixed stationside. In such configuration, words included in thenon-corresponding-to-position information can be easily extracted to beused for the character recognition processing.

The mobile terminal device in accordance with the present invention is amobile terminal device for communicating information with a fixedstation device that executes character recognition processing torecognize a character or a character string included in an image,characterized by comprising imaging means for imaging an image, positionmeasuring means for measuring a shooting position of an image to obtainshooting position information indicating the shooting position,direction detecting means for detecting a shooting direction of an imageto obtain shooting direction information indicating the shootingdirection, information transmitting means on mobile terminal side fortransmitting the shooting position information, the shooting directioninformation, and an image imaged by the imaging means via acommunication network to the fixed station device, and informationreceiving means on mobile terminal side for receiving from the fixedstation device information of a character or a character string obtainedby the fixed station device through the character recognition processingfor the image using the shooting position information and the shootingdirection information. According to such structure, since the mobileterminal device need not include the position corresponding informationstorage means and the character recognizing means, the configuration ofthe mobile terminal device can be simplified.

The fixed station device in accordance with the present invention is afixed station device for communicating information with a mobileterminal device, comprising information receiving means on fixed stationside for receiving from the mobile terminal device an image shot by themobile terminal device, the shooting position information indicating ashooting position of the image, and the shooting direction informationindicating a shooting direction of the image; position correspondinginformation storage means for storing position corresponding informationwhich is words associated with respective positional informationindicating positions of respective places, position correspondinginformation extracting means for determining, according to the shootingposition information and the shooting direction information that arereceived from the mobile terminal device, a range of an object shot bythe mobile terminal device, and extracting from the positioncorresponding information storage means position correspondinginformation associated with positions included in the range; characterrecognizing means for recognizing a character or a character stringincluded in the image received from the mobile terminal device, usingthe position corresponding information extracted by the positioncorresponding information extracting means; and information transmittingmeans on fixed station side for transmitting information of a characteror a character string recognized by the character recognizing means viaa communication network to the mobile terminal device. In suchconfiguration, since the mobile terminal device need not include theposition corresponding information storage means and the characterrecognizing means, the configuration of the mobile terminal device canbe simplified.

Also, the fixed station device may comprisenon-corresponding-to-position information storage means for storingtherein non-corresponding-to-position information that is words notassociated with positional information and non-corresponding-to-positioninformation extracting means for extractingnon-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means, wherein the characterrecognizing means recognizes a character or a character string includedin the image using the position corresponding information extracted bythe position corresponding information extracting means and thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means. In suchstructure, the configuration of the mobile terminal device can besimplified and the character recognition accuracy can be improved for acharacter string or the like not directly related to positionalinformation.

Moreover, the mobile terminal device may be a mobile terminal device forcommunicating information with a fixed station device and executingcharacter recognition processing to recognize a character or a characterstring included in an image, comprising imaging means for shooting animage, position measuring means for measuring a shooting position of animage to obtain shooting position information indicating the shootingposition, direction detecting means for detecting a shooting directionof an image to obtain shooting direction information indicating theshooting direction, information transmitting means on mobile terminalside for transmitting the shooting position information and the shootingdirection information via a communication network to the fixed stationdevice, information receiving means on mobile terminal side forreceiving from the fixed station device non-corresponding-to-positioninformation that is words not associated with respective positionalinformation indicating positions of respective places and that isdetermined by the fixed station device using the shooting positioninformation and the shooting direction information, and characterrecognizing means for recognizing a character or a character stringincluded in the image imaged by the imaging means, using thenon-corresponding-to-position information received by the informationreceiving means on mobile terminal side. In such configuration, sincethe retrieval processing of the non-corresponding-to-positioninformation is executed in the fixed station, the processing load on themobile terminal device is mitigated and the mobile terminal device neednot have high computation performance. Additionally, since the databasestoring the non-corresponding-to-position information is controlled onthe fixed station side in a unified manner, maintenance is facilitatedand maintainability can be improved.

Furthermore, the mobile terminal device may comprise positioncorresponding information storage means on mobile terminal side forstoring position corresponding information that is words associated withrespective positional information indicating positions of respectiveplaces and position corresponding information extracting means on mobileterminal side for determining, according to the shooting positioninformation and the shooting direction information, a range of ashooting object of the imaging means, and extracting from the positioncorresponding information storage means on mobile terminal side positioncorresponding information associated with positions included in therange, wherein the character recognizing means recognizes a character ora character string included in the image using thenon-corresponding-to-position information received by the informationreceiving means on mobile terminal side and the position correspondinginformation extracted by the position corresponding informationextracting means on mobile terminal side. In such structure, bytransmitting the non-corresponding-to-position information, the volumeof data transmitted from the fixed station device to the mobile terminaldevice can be reduced.

Also, a fixed station device for communicating information with a mobileterminal device that executes character recognition processing torecognize a character or a character string included in an imagecomprises information receiving means on fixed station side forreceiving from the mobile terminal device the shooting positioninformation of an image that is shot by the mobile terminal device andthat indicates a shooting position of the image and the shootingdirection information indicating a shooting direction of the image,position corresponding information storage means on fixed station sidefor storing position corresponding information that is words associatedwith respective positional information indicating positions ofrespective places, position corresponding information extracting meanson fixed station side for determining, based on the shooting positioninformation and the shooting direction information that are received bythe information receiving means on fixed station side, a range of anobject shot by the mobile terminal device, and extracting from theposition corresponding information storage means on fixed station sideposition corresponding information associated with positions included inthe range; non-corresponding-to-position information storage means forstoring non-corresponding-to-position information that is words notassociated with positional information, non-corresponding-to-positioninformation extracting means for extractingnon-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means on fixed station side, andinformation transmitting means on fixed station side for transmittingthe non-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means via acommunication network to the mobile terminal device. According to suchconfiguration, since the retrieval processing of thenon-corresponding-to-position information is executed in the fixedstation, the processing load on the mobile terminal device is mitigatedand the mobile terminal device need no have high computationperformance. Additionally, since the database storing thenon-corresponding-to-position information is controlled on the fixedstation side in a unified manner, maintenance is facilitated andmaintainability can be improved.

Moreover, the information transmitting means on fixed station side maytransmit the non-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means and theposition corresponding information extracted by the positioncorresponding information extracting means on fixed station side.According to such structure, compared with a situation in which only thenon-corresponding-to-position information is transmitted, the processingload on the mobile terminal device can be further mitigated and themobile terminal device need not have high computation performance.

A character recognition method in accordance with the present inventionis characterized in that imaging means shoots an image, positionmeasuring means measures a shooting position of an image to obtainshooting position information indicating the shooting position,direction detecting means detects a shooting direction of an image toobtain shooting direction information indicating the shooting direction,position corresponding information extracting means determines, based onthe shooting position information and the shooting directioninformation, a range of a shooting object of the imaging means; theposition corresponding information extracting means extracts, from theposition corresponding information storage means storing thereinposition corresponding information that is words associated withrespective positional information indicating positions of respectiveplaces, position corresponding information associated with positionsincluded in the range; and character recognizing means recognizes, usingthe position corresponding information extracted by the positioncorresponding information extracting means, a character or a characterstring included in the image imaged by the imaging means.

Also, in the character recognition method non-corresponding-to-positioninformation extracting means may extract, fromnon-corresponding-to-position information storage means storing thereinnon-corresponding-to-position information that is words not associatedwith positional information, non-corresponding-to-position informationaccording to the position corresponding information extracted by theposition corresponding information extracting means, and the characterrecognizing means recognizes a character or a character string includedin the image using the position corresponding information extracted bythe position corresponding information extracting means and thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means. In suchstructure, since non-geographical non-corresponding-to-positioninformation related to words extracted from the position correspondinginformation storage means can be read to be used for characterrecognition, recognition accuracy can be improved for character stringsor the like that cannot be directly associated with positionalinformation such as a person's name, an item name, or a date.

Furthermore, the character recognition method may be a characterrecognition method of recognizing a character or a character stringincluded in an image, for use in a mobile communication system in whicha mobile terminal device and a fixed station device communicate via awireless transmission path with each other, wherein the mobile terminaldevice shoots an image, the mobile terminal device measures a shootingposition of an image to obtain shooting position information indicatingthe shooting position, the mobile terminal device detects a shootingdirection of an image to obtain shooting direction informationindicating the shooting direction, the mobile terminal device transmitsthe shooting position information, the shooting direction information,and an image imaged by the imaging means via a communication network tothe fixed station device; the fixed station device determines, based onthe shooting position information and the shooting direction informationthat are received from the mobile terminal device, a range of a shootingobject of the mobile terminal device; the fixed station device extracts,from position corresponding information storage means storing positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places,position corresponding information associated with positions included inthe range; the fixed station device recognizes, using the extractedposition corresponding information, a character or a character stringincluded in the image received from the mobile terminal device; and thefixed station device transmits information of a character or a characterstring via a communication network to the mobile terminal device. Insuch structure, since the mobile terminal device need not include theposition corresponding information storage means and the characterrecognizing means, the configuration of the mobile terminal device canbe simplified.

Furthermore, in the character recognition method the mobile terminaldevice may receive information of a character or a character stringrecognized by the fixed station device, via a communication network fromthe fixed station device; and the mobile terminal device outputsinformation of the character or the character string. In such structure,the result information of the character recognition can be supplied tothe user of the mobile terminal by displaying the information on adisplay module and/or by producing sound and voice.

Also, the character recognition method may be a character recognitionmethod of recognizing a character or a character string included in animage, for use in a mobile communication system in which a mobileterminal device and a fixed station device communicate via a wirelesstransmission path with each other, wherein the mobile terminal deviceimages an image, the mobile terminal device measures a shooting positionof an image to obtain shooting position information indicating theshooting position, the mobile terminal device detects a shootingdirection of an image to obtain shooting direction informationindicating the shooting direction, the mobile terminal device transmitsthe shooting position information and the shooting direction informationvia a communication network to the fixed station device, the fixedstation device determines, based on the shooting position informationand the shooting direction information that are received from the mobileterminal device, a range of a shooting object of the mobile terminaldevice; the fixed station device extracts, from position correspondinginformation storage means on fixed station side storing positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places,position corresponding information associated with positions included inthe range; the fixed station device extracts, fromnon-corresponding-to-position information storage means storingnon-corresponding-to-position information that is words not associatedwith positional information, non-corresponding-to-position informationaccording to the position corresponding information extracted from theposition corresponding information storage means on fixed station side;the fixed station device transmits the extractednon-corresponding-to-position information via a wireless transmissionnetwork to the mobile terminal device; and the mobile station devicerecognizes a character or a character string included in the image shotby the mobile terminal device, using the non-corresponding-to-positioninformation received from the fixed station device. In suchconfiguration, since the retrieval processing of thenon-corresponding-to-position information is executed in the fixedstation, the processing load on the mobile terminal device is mitigatedand the mobile terminal device need not have high computationperformance. Additionally, since the database storing thenon-corresponding-to-position information is controlled on the fixedstation side in a unified manner, maintenance is facilitated andmaintainability can be improved.

A character recognition program in accordance with the present inventioncauses a computer to execute processing for determining, based onshooting position information indicating a shooting position of an imageand shooting direction information indicating a shooting direction of animage, a range of a shooting object; processing for extracting, fromposition corresponding information storage means storing positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places,position corresponding information associated with positions included inthe range; and character recognition processing for recognizing, usingthe extracted position corresponding information, a character or acharacter string included in the image.

Also, the character recognition program may make a computer executeprocessing for extracting, from non-corresponding-to-positioninformation storage means storing non-corresponding-to-positioninformation which is words not associated with positional information,non-corresponding-to-position information according to the positioncorresponding information extracted from the position correspondinginformation storage means; and processing for recognizing a character ora character string included in the image using the positioncorresponding information extracted from the position correspondinginformation storage means and the non-corresponding-to-positioninformation extracted from the non-corresponding-to-position informationstorage means. In such structure, since non-geographicalnon-corresponding-to-position information related to words extractedfrom the position corresponding information storage means can be read tobe used for character recognition, recognition accuracy can be increasedfor character strings or the like which cannot be directly associatedwith positional information such as a person's name, an item name, or adate.

The information communication program in accordance with the presentinvention is an information communication program in a mobile terminaldevice communicating information with a fixed station device thatexecutes character recognition processing to recognize a character or acharacter string included in an image, the program causing a computer toexecute processing for transmitting shooting position informationindicating a shooting position of an image, shooting directioninformation indicating a shooting direction of an image, and an imagevia a wireless transmission network to the fixed station device; andprocessing for receiving from the fixed station device information of acharacter or a character string obtained by the fixed station devicethrough the character recognition processing for the image using theshooting position information and the shooting direction information.According to such structure, since the mobile terminal device need notinclude the position corresponding information storage means and thecharacter recognizing means, the configuration of the mobile terminaldevice can be simplified.

Also, the character recognition program may be a character recognitionprogram in a fixed station device for communicating information with amobile terminal device, the program causing a computer to executeprocessing for receiving from the mobile terminal device the image shotby the mobile terminal device, the shooting position informationindicating a shooting position of the image, and the shooting directioninformation indicating a shooting direction of the image; processing fordetermining, according to the shooting position information and theshooting direction information which are received from the mobileterminal device which are received from the mobile terminal device, arange of an object shot by the mobile terminal device; processing forextracting, from position corresponding information storage meansstoring position corresponding information which is words associatedwith respective positional information indicating positions ofrespective places, position corresponding information associated withpositions included in the range; processing for recognizing a characteror a character string included in the image received from the mobileterminal device, using the extracted position corresponding information;and processing for transmitting information of a character or acharacter string via a communication network to the mobile terminaldevice. According to such structure, since the mobile terminal deviceneed not include the position corresponding information storage meansand the character recognizing means, the configuration of the mobileterminal device can be simplified.

Furthermore, the character recognition program may be a characterrecognition program in a mobile terminal device for communicatinginformation with a fixed station device and executing characterrecognition processing to recognize a character or a character stringincluded in an image, the program causing a computer to executeprocessing for transmitting shooting position information indicating ashooting position of an image and shooting direction informationindicating a shooting direction of an image, via a communication networkto the fixed station device; processing for receiving from the fixedstation device non-corresponding-to-position information which is wordsnot associated with respective positional information indicatingpositions of respective places and which is determined by the fixedstation device using the shooting position information and the shootingdirection information; and processing for recognizing a character or acharacter string included in the image shot by the mobile terminaldevice, using the non-corresponding-to-position information receivedfrom the fixed station device. According to such configuration, sincethe retrieval processing of the non-corresponding-to-positioninformation is executed in the fixed station, the processing load on themobile terminal device is mitigated and the mobile terminal device neednot have high computation performance. Additionally, since the databasestoring the non-corresponding-to-position information is controlled onthe fixed station side in a unified manner, maintenance is facilitatedand maintainability can be improved.

The information extraction program in accordance with the presentinvention is an information extraction program in a fixed station devicefor communicating information with a mobile terminal device thatexecutes character recognition processing to recognize a character or acharacter string included in an image, causing a computer to executeprocessing for receiving from the mobile terminal device shootingposition information indicating a shooting position of an image shot bythe mobile terminal device and shooting direction information indicatinga shooting direction of the image; processing for determining, accordingto the shooting position information and the shooting directioninformation which are received from the mobile terminal device, a rangeof an object shot by the mobile terminal device; processing forextracting, from position corresponding information storage means onfixed station side storing position corresponding information which iswords associated with respective positional information indicatingpositions of respective places, position corresponding informationassociated with positions included in the range; processing forextracting, from non-corresponding-to-position information storage meansstoring non-corresponding-to-position information which is words notassociated with positional information, non-corresponding-to-positioninformation according to the position corresponding informationextracted from the position corresponding information storage means onfixed station side; and processing for transmitting the extractednon-corresponding-to-position information via a communication network tothe mobile terminal device. In such configuration, since the retrievalprocessing of the non-corresponding-to-position information is executedin the fixed station, the processing load on the mobile terminal deviceis mitigated and the mobile terminal device need not have highcomputation performance. Additionally, since the database storing thenon-corresponding-to-position information is controlled on the fixedstation side in a unified manner, maintenance is facilitated andmaintainability can be improved.

Advantages of the Invention

In accordance with the present invention, by using both of the shootingposition information and the shooting direction information, only theword knowledge likely to be contained in the scene image with a highprobability is extracted from the position corresponding informationstorage means (such as a word dictionary and a map database), and hencethe word knowledge to be used for the character recognition can beefficiently limited, compared to a case where the word knowledge islimited using only the positional information. Therefore, the characterstring recognition speedup and the recognition accuracy improvement canbe achieved. Consequently, words possibly contained in the scene imageshot by a mobile camera can be efficiently extracted by using a worddictionary or a map database.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1

Next, description will be given of a first embodiment of the presentinvention by referring to drawings. FIG. 1 is a block diagram showing anexample of structure of a character recognition device in accordancewith the present invention. As shown in FIG. 1, the characterrecognition device includes positional information acquiring means 101,directional information acquiring means 102, a map database 103,character string recognizing means 104, and image input means 106.

The image input means 106 is implemented by, for example, a mobile videocamera. The image input means 106 shoots a scene image (input image) anddelivers image data to the character string recognizing means 104.

The positional information acquiring means 101 is implemented by, forexample, a GPS receiver. The means 101 measures a position of a device(e.g., the image input means 106) to obtain positional information(latitude and longitude) and feeds the information to the characterstring recognizing means 104. That is, the means 101 outputs shootingposition information indicating a shooting position of the scene image.

The directional information acquiring means 102 is implemented by, forexample, a gyroscope. The means 102 detects a direction of a device(e.g., the image input means 106) to obtain information of a directionof the device (directional information such as east, west, south, ornorth) and feeds the information to the character string recognizingmeans 104. That is, the means 102 outputs shooting direction informationindicating a shooting direction of the scene image.

The map database 103 is implemented by, for example, a storage devicesuch as a magnetic disk device (not shown). The map database 103 storestherein transportation network information regarding transport networkssuch as roads and railroads, place name information indicating placenames and positions of regions, building information such as names,structures, and addresses of buildings, and store information indicatingnames, telephone numbers, and addresses of stores. Additionally, the mapdatabase 103 stores word information such as traffic networkinformation, place information, building information, and storeinformation in association with positional information (information suchas latitude and longitude).

The character string recognizing means 104 is implemented by, forexample, a computer (not shown) operating according to a characterrecognition program, which will be described later. The means 104identifies, according to the current position information and thedirectional information of the character recognition device, a range(shooting range) of a place shot in the scene image. The range of theplace shot in the scene image determined by the character stringrecognizing device is simply described as a particular rangehereinbelow.

The character string recognizing means 104 extracts information such asa store name, a building name, and a place name associated withlocations in the particular range, from the map database. The means 104conducts character recognition using the extracted information such as astore name, a building name, and a place name. Also, the means 104conducts character recognition using also information of names of lands,intersections, and stations connected via a transportation network and arailroad network shot in the scene image. Incidentally, in theembodiment, word information such as a place name and a building namethat the means 104 uses for the character recognition is described asword knowledge.

The imaging means is realized by the image input means 106. The positionmeasuring means is implemented by the positional information acquiringmeans 101. The direction detecting means is realized by the directionalinformation acquiring means 102. The position corresponding informationstorage means is implemented by the map database 103. The positioncorresponding information extracting means and the character recognizingmeans are realized by the character string recognizing means 104.

Furthermore, in the embodiment, a storage device (not shown) of thecomputer realizing the character string recognizing means 104 has storedvarious programs to execute the character recognition processing. Forexample, the storage device of the computer has stored a characterrecognition program to cause the computer to execute processing todetermine a range of the shooting object according to the shootingposition information indicating an image shooting position and theshooting direction information indicating an image shooting direction,processing to extract position corresponding information associated withpositions contained in the range from the position correspondinginformation storage means storing therein position correspondinginformation items that are words associated with respective positionalinformation indicating positions of respective places, and characterrecognition processing to recognize characters or character stringscontained in an image by use of the extracted position correspondinginformation.

Next, operation will be described. FIG. 2 is a flowchart showing anexample of the character recognition processing which the characterrecognition device executes. The image input means 106 shoots a sceneimage to deliver image data to the character string recognizing means104 (step S10). The positional information acquiring means 101 measuresthe location of the device to acquire the current position informationto output the information to the character string recognizing means 104(step S11). The directional information acquiring means 102 detects thedirection of the device to acquire directional information and deliversthe information to the means 104 (step S12).

The character string recognizing means 104 refers to the map database103 based on the current position information and the directionalinformation to restrict words to be used in the recognition processingand extracts word knowledge from the map database 103. As a result, wordknowledge such as place names and building names are extracted.Thereafter, the means 104 reads character strings from the scene imageby using the extracted word knowledge to thereby conduct the characterrecognition (step S13).

When the character recognition is finished, the character stringrecognizing means 104 determines whether an indication of the processingend is inputted by a user or the like (step S14). For example, the means104 displays, after the character recognition is finished, the result ofthe character recognition on a display module (not shown) or the likeand displays an end/continuance indication input screen of characterrecognition processing. And the user inputs, to terminate the characterrecognition processing, an indication of the processing end from aninput module (not shown) or the like. Incidentally, the means 104 maysupply the character recognition result information through voice andsound by outputting the information to a voice and sound output module(not shown) or the like, not by displaying the information.

If it is determined that a processing end indication is inputted, thecharacter recognition device terminates the character recognitionprocessing. If it is not determined that a processing end indication isinputted, the character recognition device returns again to step S10 torepeatedly execute the processing of steps S10 to S14.

As above, according to the embodiment, the character recognition deviceextracts word knowledge from the map database 103 based on thepositional information and the directional information acquired by thepositional information acquiring means 101 and the directionalinformation acquiring means 102. Thereafter, the character recognitiondevice executes the character recognition processing by use ofinformation such as region names, building names, store names, storeaddresses, and store telephone numbers associated with a region in thescene image and information such as place names of regions connected viaa transportation network to the associated region.

By use of both of the positional information and the directionalinformation, only the word knowledge such as place names, road names,building names, and store names that will be contained in the sceneimage with a high probability are extracted from the map database 103,and hence the word knowledge to be used for the knowledge recognitioncan be efficiently restricted in comparison with the case where wordknowledge is extracted using only the positional information.Consequently, the speedup of recognition and the recognition accuracyimprovement can be achieved in the character string recognition.

FIG. 3 is an explanatory diagram showing an example of a particularrange that the character recognition device determines. FIG. 3( a) is anexplanatory diagram showing an example of a particular range that thecharacter recognition device determines in accordance with the presentinvention. In FIG. 3( a) the black point indicates the current position.Also, FIG. 3( b) is an explanatory diagram showing an example of aparticular range which the conventional character recognition device(e.g., a server of the system described in patent document 2)determines. The region in a scene image is in general an area of asector shape on a plane. When word knowledge is extracted from a mapdatabase based only on positional information such as the GPSinformation, the character recognition must be conducted using all wordsrelated to the circular range whose center is the current position ofthe device as shown in FIG. 3( b).

In accordance with the embodiment, as shown in FIG. 3( a), by using thedirectional information in addition to the positional information, it ispossible to extract only the word knowledge related to the sector rangeshot in the scene image. Therefore, by use of the directionalinformation, the character recognition can be conducted without usinginformation related to regions outside the scene image range, and hencethe words can be efficiently restricted as compared with theconventional case where only the positional information is used.Therefore, the speedup of the recognition speed and the improvement ofrecognition accuracy can be achieved when compared with the conventionalcase where only the positional information is used.

Moreover, with the transportation network information stored in the mapdatabase 103, place names of places outside the sector range shown inFIG. 3( a) such as place names of adjacent zones that can be describedin directional signs can also be utilized in the character recognitionprocessing.

Next, description will be given of a concrete example of the embodiment.The description here is given by use of an example of a case where thecharacter recognition device includes a GPS receiver as the positionalinformation acquiring means 101, a gyroscope as the directionalinformation acquiring means 102, a magnetic disk device as the mapdatabase 103, a computer as the character string recognizing means 104,and a mobile video camera as the image input means 106.

The video camera shoots a scene image to output image data to thecomputer. The GPS receiver measures the current position of the deviceaccording to a received GPS signal to acquire latitude and longitude(positional information) and delivers the information to the computer.

The gyroscope detects a direction of the device (the video camera inthis example) to acquire directional information and delivers theinformation to the computer. For example, assume that the direction ofthe directional information is expressed by a numeric value, and thenorth, the east, the south, and the west are represented as “0”, “90”,“180”, and “270”. In this case, if the device faces southwest, thegyroscope outputs a value “255” as the directional information. In thisregard, the representing method of the directional information is notlimited to that of this example, and it is also possible to use anyother representing method capable of uniquely determining the direction.

The map database 103 implemented by the magnetic disk device storestherein transportation network information, place name information,building information, and store information. FIGS. 4 to 7 areexplanatory diagrams showing examples of the contents of thetransportation network information, the place name information, thebuilding information, and the store information.

FIG. 4 is an explanatory diagram showing an example of thetransportation network information regarding roads. As shown in FIG. 4,the map database 103 stores therein, for example, information such asintersection names 601, intersection addresses (addresses 602 andlatitude/longitude 603), adjacent intersection names/directions 604 asthe transportation network information regarding roads. Furthermore, themap database 103 may store therein, for example, information such as aroad name connecting intersections as the transportation networkinformation.

FIG. 5 is an explanatory diagram showing an example of the place nameinformation. FIG. 5( a) is an explanatory diagram showing an example ofa place name table 706 the map database 103 stores as the place nameinformation. FIG. 5( b) is an explanatory diagram showing an example ofa two-dimensional map 707 the map database 103 stores as the place nameinformation. As shown in FIG. 5( b), the map database 103 stores, as theplace name information, the two-dimensional map 707 labeled (C1 to C3)for regions 703 to 705. Furthermore, the map database 103 stores theplace name table 706 indicating a correspondence between the place name701 and the map label value 702 as shown in FIG. 5( a). Also, when the xand y coordinate values of the two-dimensional map 707 are associatedwith latitude/longitude, it is possible to know the name of the currentposition from the information of latitude/longitude.

FIG. 6 is an explanatory diagram showing an example of the buildinginformation. The map database 103 stores, for example, informationregarding various building attributes such as a building name 801, anaddress 802, latitude/longitude 803, and height 804 of each building asthe building information.

FIG. 7 is an explanatory diagram showing an example of the storeinformation. The map database 103 stores, for example, information suchas a store name 901, an address 902, a building name/floor 903, and atelephone number 904 as the store information.

Next, description will be given of a method of extracting a store name,a building name, and a place name when the current positionalinformation and the current directional information to be used in therecognition are inputted respectively from the GPS receiver and thegyroscope to the computer.

First, by referring to drawings, description will be given of a methodof extracting a place name. FIG. 8 is an explanatory diagram displayingthe current position, the current direction, and the shooting range ofthe character recognition device at a certain point of time on a map. Inthe example, description will be given of a case where the currentposition is a point 1001, the current direction is “facing north”, andan angle of view of a scene image is 90 degrees, and the shooting rangeis a shaded area 1002. If a store signboard is included in the sceneimage, an address name of the shaded area 1002 will exist in the storesignboard with a high probability. In this situation, the characterrecognition device refers to the place name information stored in themap database 103 to extract the place name of an area corresponding tothe shaded area 1002.

For example, when the point 1001 is a point 708 of FIG. 5( b) and theshaded range 1002 is a shaded range 709 of FIG. 5( b), the characterrecognition device determines, according to the two-dimensional map 707shown in FIG. 5( b), that a region shot in the scene image is a rangewith a label value of “C2”. Also, the character recognition deviceextracts, according to the place name table 706 shown in FIG. 5( a), aplace name of “Kinza 2-chome” corresponding to the label value “C2”.That is, first, a word of a place name such as “Kinza 2-chome” or“Kinza” is extracted as a word to be used for the character recognition.

Furthermore, a name of a place in the neighborhood may be on a trafficsign. Therefore, the character recognition device refers to thetransport network information shown in FIG. 4 to extract a place name ofa region that will be reached if the car turns left at the intersection,runs straight through the intersection, or turns right at theintersection. For example, if it is assumed that a point of eastlongitude “a2”/north latitude “b2” is included in the shooting range ofthe scene image at a point of time, the character recognition devicedetermines according to the transportation network information shown inFIG. 4 that “Kinza 2-chome intersection” is included in the shootingrange. Thereafter, the character recognition device extracts, accordingto the transportation network information shown in FIG. 4, intersectionnames of “Kinza 1-chome” and “Kinza 3-chome” existing in theneighborhood, addresses of the intersections, and a place name of anintersection appearing thereafter.

Next, description will be given of a method of extracting a buildingname, a store name, and a store telephone number by referring todrawings. A building name and a store name may be described on a storesignboard or the like. Therefore, the character recognition deviceextracts information of buildings included in the latitude/longitude ofthe shaded area 1002 according to the latitude/longitude 803 of thebuilding information shown in FIG. 6. That is, the character recognitiondevice extracts a building name 801 and an address 802 of a buildingaccording to the latitude/longitude 803. Also, the device compares theextracted building name 801 with the store information shown in FIG. 7to extract information of a store name 901 and a telephone number 904 inthe building.

The character string recognizing means (a computer in this example) 104conducts character recognition for the inputted scene image by using theword knowledge such as the place names and the store names extracted inthe method described above. The means 104 conducts the characterrecognition by using, for example, as a character recognition algorithmemploying word knowledge, a dynamic planning method or a Hidden MarkovModel (HMM) method generally used for mail address recognition.

For example, the character string recognizing means 104 assumes, usingthe dynamic planning method or the HMM method, that each of the variousfigures included in the input image as one character and then recognizesa character for each figure. Then the means 104 combines the charactersrecognized for the respective figures in various ways and compares thecombined character string with the word knowledge to thereby conduct thecharacter recognition at a word level.

If the word knowledge is not employed, a word cannot be correctlyrecognized unless the recognition is correctly conducted for eachcharacter. However, by use of the word knowledge, a slight recognitionerror of each character can be corrected, and hence the characterrecognition accuracy can be improved.

Next, description will be given of an advantage of the present inventionby referring to drawings. In the present invention, as shown in FIG. 8,the character recognition processing can be achieved without using theword knowledge such as place names, building names, and store namesbelonging to a region outside the shooting range, the shaded area 1002.For example, as shown in FIG. 8, a building 1008 exists relatively nearthe current position 1001 but in the reverse direction of the shooting,and hence the character recognition device can conduct the characterrecognition processing by assuming that the word knowledge regarding abuilding name of the building 1008, store names, store addresses,telephone numbers, and the like in the building is not the processingobject.

As above, in accordance with the present invention, the word knowledgerequired for the processing can be restricted in comparison with theconventional case in which word knowledge is extracted according only tothe positional information. As the amount of word knowledge reduces, thecharacter recognition processing can be executed at a higher speed, andhence the present invention can accelerate the recognition processing.

Incidentally, in the description of the concrete example, the imageinput means 106 is a mobile video camera. However, the means 106 may beany other imaging means. For example, the means 106 may be a portabletelephone or a Personal Digital Assistant (PDA) including a camera, thetelephone or the PDA being carried about by a user. Also, the means 106may be a digital camera or a vehicle mounted camera fixed on a mobilevehicle. Additionally, the image shot by the means 106 may be a stillimage or a moving image.

Embodiment 2

Next, description will be given of a second embodiment of the presentinvention by referring to drawings. FIG. 9 is a block diagram showinganother example of the configuration of the character recognitiondevice. As shown in FIG. 9, the device includes a positional informationacquiring means 201, a directional information acquiring means 202, amap database 203, a character string recognizing means 204, a documentdatabase 205, and an image input means 206.

In the embodiment, the character recognition device includes, inaddition to the constituent components shown in the first embodiment, adocument database 205. In the embodiment, the device differs from thefirst embodiment in that the device conducts the character recognitionprocessing using words included in the information stored in thedocument database in addition to the map database 203. In thisconnection, functions and implementing methods of the positionalinformation acquiring means 201, the directional information acquiringmeans 202, the map database 203, and the image input means 206 aresimilar to those of the positional information acquiring means 101, thedirectional information acquiring means 102, the map database 103, andthe image input means 106 shown in the first embodiment.

The document database 205 is realized by a storage device, for example,a magnetic disk device (not shown) or the like. The document database205 stores therein text information not directly associated with thepositional information such as latitude/longitude. For example, thedocument database 205 stores a set of text information items such as“Painter A's private exhibition is held at a special events hall of the8th floor of department store B from 3rd to 6th October” and “A goodseller of bakery C is a cream bun”. The text information stored in thedocument database 205 will be referred to as document information(non-corresponding-to-position information) hereinbelow.

As will be described later, the character recognition device of theembodiment conducts character recognition by using also the wordknowledge included in the document information. For example, the deviceconducts character recognition using also the word knowledge such as“3rd October” and “Painter A” contained in the document information“Painter A's private exhibition is held at a special events hall of the8th floor of department store B from 3rd to 6th October”.

The character string recognizing means 204 is realized, for example, bya computer (not shown) or the like operating according to a characterrecognition program. The character recognition device 204 determines arange of a place shot in the scene image, according to the currentposition information and the directional information of the device. Thedevice 204 extracts from the map database 203 information such as storenames, building names, and place names associated with locations in theparticular range.

Furthermore, the character string recognizing means 204 conducts notonly the character recognition using the word knowledge extracted fromthe map database 203, but extracts from the document database 205document information related to the word knowledge extracted from themap database 203. Also, the character string recognizing means 204conducts the character recognition processing also using the documentinformation in addition to the word knowledge extracted from 203.

By extracting the related document information from the documentdatabase 205 and by using the information as word knowledge, thecharacter string recognition processing can be conducted by use of theextracted document information. Also, if the extracted documentinformation is not in word format, the character string recognizingmeans 204 conducts a syntactic analysis to divide the documentinformation into word units, and then the information can be used in thecharacter string recognition processing.

The non-corresponding-to-position information storing means is realizedby the document database 205. The non-corresponding-to-positioninformation extracting means is realized by the character stringrecognizing means 204.

Moreover, in the embodiment, the storage device (not shown) of thecomputer implementing the character string recognizing means 204 hasstored various programs to execute the character recognition processing.For example, the storage device of the computer stores therein acharacter recognition program to make a computer execute processing fordetermining, according to shooting position information indicating ashooting position of an image and shooting direction informationindicating a shooting direction of an image, a range of an object ofimaging; processing for extracting, from position correspondinginformation storage means storing position corresponding informationwhich is words associated with respective positional informationindicating positions of respective places, position correspondinginformation associated with positions included in the range; processingfor extracting, from non-corresponding-to-position information storagemeans storing therein non-corresponding-to-position information which iswords not associated with positional information,non-corresponding-to-position information according to the positioncorresponding information extracted from the position correspondinginformation storage means; and processing for recognizing a character ora character string included in the image using the positioncorresponding information extracted from the position correspondinginformation storage means and the non-corresponding-to-positioninformation extracted from the non-corresponding-to-position informationstorage means.

Next, the operation will be described. FIG. 10 is a flowchart showinganother example of the processing progress of the character recognitionprocessing executed by the character recognition device. In FIG. 10, theprocessing from step S20 to step S22 is similar to that from step S10 tostep S12 shown in the first embodiment.

The character string recognizing means 204 refers to the map database203 according to the current positional information and the directionalinformation to extract word knowledge from the map database 203 (stepS23). The recognizing means 204 makes a search through the documentdatabase 205 using the extracted word knowledge as a key word to extractdocument information from the database 205 (step S24). Thereafter, byusing the word knowledge extracted from the map database 203 and thedocument information extracted from the document database 205, therecognizing means 204 reads character information from the scene imageto thereby conduct the character recognition (step S25).

When the character recognition is finished, the recognizing means 204determines whether an indication of the processing end is inputted by auser or the like (step S26). If it is determined that the indication ofthe processing end is inputted, the character recognition deviceterminates the character recognition processing. If it is not determinedthat the indication of the processing end is inputted, the characterrecognition device returns again to the processing of step S20 torepeatedly execute the processing from step S20 to step S26.

In this regard, the character string recognizing means 204 may display,for example, information resultant from the character recognition on adisplay module (not shown) to thereby supply the information to theuser. Also, the recognizing means 204 may deliver, for example, thecharacter recognition result information to a sound and voice outputmodule (not shown) to thereby supply the information to the user.

As above, according to the embodiment, the character recognition deviceextracts, according to the positional information and the directionalinformation, related document information from the document database205, the related document information which is not directly associatedwith the positional information and is related to the word knowledgeextracted from the map database 203. Then the character recognitiondevice conducts character recognition using the related documentinformation. Therefore, it is possible for the device to extract, fromthe document database 205, the non-geographical document informationrelated to the words extracted from the map database 203 and to use thedocument information for the character recognition, and hence therecognition accuracy can be increased for a character string such as aname of a person, an item name, or a date which cannot be directlyassociated with the positional information.

Next, description will be given of a concrete example of the embodiment.In this situation, description will be given of an example of a case inwhich the character recognition device includes a GPS receiver as thepositional information acquiring means 201, a gyroscope as thedirectional information acquiring means 202, a magnetic disk device formaps as the map database 203, a magnetic disk device for documents asthe document database 205, a computer as the character stringrecognizing means 204, and a vehicle mounted camera as the image inputmeans 206, the camera being fixed onto a mobile vehicle.

The GPS receiver and the gyroscope are similar to the concrete exampleshown in the first embodiment. Also, the magnetic disk device for mapsis similar to the magnetic disk device of the concrete example shown inthe first embodiment.

The on-vehicle camera shoots a scene and outputs image data to thecomputer. The magnetic disk device for documents has stored variousdocument information. In this regard, the document information may bedata described in a document format or in a description format (e.g.,data of a set of words) in which key words are related with each otherin a network contour.

The computer refers to, according to the current positional informationand the directional information inputted from the GPS receiver and thegyroscope, the magnetic disk devices for maps and documents to readcharacters from the scene image. Incidentally, the method for thecomputer to extract word knowledge to be used for character recognitionfrom the magnetic disk device for maps is similar to the concreteexample shown in the first embodiment.

Next, by referring to drawings, description will be given of a methodfor the computer to extract word knowledge to be used for characterrecognition from the magnetic disk for documents. FIG. 11 is anexplanatory diagram showing an example of a hanging screen of adepartment store shot by a camera mounted on a vehicle that is runningthrough a shopping quarter. As shown in FIG. 11, proper nouns such as“painter A” are described on the hanging screen in this example. In thiscase, since the word “painter A” is information not directly related tothe positional information such as latitude/longitude, the computer(character string recognizing means 204) cannot extract word knowledgeregarding “painter A” from the magnetic disk device for maps (mapdatabase 203).

In the example, the computer extracts “department store B” shot in thescene image from the magnetic disk device for maps, based on the wordknowledge stored in the magnetic disk device for maps (map database 203)and the positional information/directional information. Then thecomputer retrieves a related document from the document informationstored in the magnetic disk device for documents, using “departmentstore B” as a key word (retrieval key).

For example, when the magnetic disk device for documents has storeddocument information of “Painter A private exhibition is held at a 8thfloor special events hall of department store B from 3rd to 6thOctober”, the computer extracts word knowledge such as “3rd October”,“6th”, “painter A”, “private exhibition”, “8th floor”, and “specialevents hall” from the document information. The computer then conductscharacter recognition using the extracted word knowledge. Therefore, thecomputer can conduct character recognition using the word knowledge“painter A”.

That is, in the concrete example, the magnetic disk device for documentshas stored, as document information, a set of words not associated withthe positional information, the set including at least one word(“department store B” in this example) equal to word knowledge stored inthe magnetic disk device for maps. And the computer extracts a set ofwords such as “painter A” including “department store B” from themagnetic disk device for documents using as a key the word knowledge(“department store B” in the example) extracted from the magnetic diskdevice for maps.

Incidentally, in this example, description has been given of a casewhere the document information stored in the magnetic disk device fordocuments is described in a document format and the word knowledge isextracted from the document information obtained through key wordretrieval. However, the document information is not restricted by thedocument format shown in the example. For example, the documentinformation stored in the magnetic disk device for documents may be datadescribed as a set of word knowledge such as “3rd October”, “6thOctober”, “painter A”, “private exhibition”, “department store B”, “8thfloor”, and “special events hall”.

Additionally, if the document information stored in the magnetic diskdevice for documents is advertisement information with a limited periodof time and news items, it is also possible to update the documentinformation according to necessity. For example, the user of thecharacter recognition device may update, using software to update adocument database, the document information stored in the magnetic diskdevice for documents. Also, for example, it is possible to update thedocument information by receiving text information via the Internet.

Embodiment 3

Next, description will be given of a third embodiment of the presentinvention by referring to drawings. FIG. 12 is a block diagram showingan example of the configuration of a mobile communication system inwhich a mobile terminal device and an in-fixed-station device conductcommunication with each other via a wireless transmission path. As shownin FIG. 12, the mobile communication system includes a mobile terminaldevice 300 a installed in a movable mobile station (e.g., a carnavigation terminal device mounted on a vehicle) and an in-fixed-stationdevice 300 b installed in a fixed station. Moreover, the mobile terminaldevice 300 a includes positional information acquiring means 301,directional information acquiring means 302, image input means 303, andinformation transmitting means 304. Furthermore, the in-fixed-stationdevice 300 b includes information receiving means 305, a map database306, and character string recognizing means 307.

Description will be given of a function of the mobile terminal device300 a. The image input means 303 is realized by a mobile video camera orthe like. The means 303 outputs scene image data to the informationtransmitting means 304.

The positional information acquiring means 301 is realized by a GPSreceiver or the like. The means 301 measures a position of the imageinput means 303 when the scene is shot to thereby obtainlatitude/longitude (positional information). Then the means 301 deliversthe obtained positional information to the information transmittingmeans 304.

The directional information acquiring means 302 is implemented by agyroscope or the like. The means 302 detects a direction (east, west,south, north) of the image input means 303 when the scene is shot tothereby obtain directional information. Then the means 302 outputs theobtained directional information to the information transmitting means304.

The information transmitting means 304 is realized by a computer (notshown) operating according to an information receiving program, whichwill be described later, a network interface module (not shown) of themobile terminal device 300 a, and the like. In this regard, theinformation receiving program is stored in a storage device (not shown)of the device 300 a. The information transmitting means 304 transmitsinformation (positional information, directional information, and imagedata) inputted from the positional information acquiring means 301, thedirectional information acquiring means 302, and the image input means303 via a wireless communication network to the information receivingmeans 305 of the in-fixed-station device 300 b.

In the embodiment, the storage device of the mobile terminal device 300a has stored various programs to communicate information with thein-fixed-station device 300 b. For example, the storage device of thedevice 300 a has stored therein an information communication program tomake a computer execute processing for transmitting shooting positioninformation indicating a shooting position of an image, shootingdirection information indicating a shooting direction of an image, andan image thus shot via a wireless transmission network to the fixedstation device and processing for receiving from the fixed stationdevice information of a character or a character string obtained by thefixed station device through the character recognition processing forthe image using the shooting position information and the shootingdirection information.

Next, description will be given of a function of the in-fixed-stationdevice (fixed station device) 300 b. The information receiving means 305is realized by a computer (not shown) operating according to a characterrecognition program, a network interface module (not shown) of thein-fixed-station device 300 b, and the like. In this regard, thecharacter recognition program is stored in a storage device (not shown)of the device 300 b. The information transmitting means 304 receives,via a wireless communication network, information (positionalinformation, directional information, and image data) transmitted fromthe information transmitting means 304. And the information receivingmeans 305 delivers the received information to the character stringrecognizing means 307.

The map database 306 is implemented by a storage device such as amagnetic disk device (not shown). The map database 306 stores thereintransportation network information indicating positions oftransportation networks, place name information indicating place namesand existing ranges of regions, building information indicating buildingnames, building structures, and building locations, and storeinformation indicating store names, store telephone numbers, and storeaddresses.

The character string recognizing means 307 is realized by a computer(not shown) or the like operating according to a character recognitionprogram. The means 307 extracts from the map database 306 word knowledgepossibly included in a scene image, according to the positionalinformation and the directional information received by the informationreceiving means 305. Then the means 307 reads, using the extracted wordknowledge, character information existing in the scene image inputtedfrom the information receiving means 305 to thereby conduct characterrecognition.

In the embodiment, the storage device of the fixed station device 300 bhas stored various programs to conduct character recognition. Forexample, the storage device of the device 300 b stores therein acharacter recognition program to cause a computer to execute processingfor receiving from the mobile terminal device the image shot by themobile terminal device, the shooting position information indicating ashooting position of the image, and the shooting direction informationindicating a shooting direction of the image; processing fordetermining, according to the shooting position information and theshooting direction information that are received from the mobileterminal device, a range of an object shot by the mobile terminaldevice; processing for extracting, from position correspondinginformation storage means storing therein position correspondinginformation that is words associated with respective positionalinformation indicating positions of respective places, positioncorresponding information associated with positions included in therange; processing for recognizing a character or a character stringincluded in the image received from the mobile terminal device, usingthe extracted position corresponding information; and processing fortransmitting information of a character or a character string thusrecognized via a communication network to the mobile terminal device.

Moreover, in the embodiment, the imaging means is realized by the imageinput means 303. The position measuring means is realized by thepositional information acquiring means 301. The direction detectingmeans is realized by the directional information acquiring means 302.Information transmitting means on mobile terminal side is realized bythe information transmitting means 304. Position correspondinginformation storage means is realized by the map database 306. Positioncorresponding information extracting means and character recognizingmeans are realized by the character string recognizing means 307.Information transmitting means on mobile station side is realized by acomputer and a network interface module of the in-fixed-station device300 b. Information receiving means on mobile station side is realized bya computer and a network interface module of the mobile terminal device300 a. Output means is realized by a computer, a display module (notshown), and a voice and sound output module (not shown) of the mobileterminal device.

Next, the operation will be described. FIG. 13 is a flowchart showing anexample of the information acquiring processing where the mobileterminal device 300 a acquires the positional information, thedirectional information, and the scene image data. Moreover, FIG. 14 isa flowchart showing an example of the information recognition processingexecuted by the in-fixed-station device 300 b.

First, description will be given of operation of the mobile terminaldevice 300 a. The image input means 303 shoots a scene to output imagedata (step S31). The positional information acquiring means 301 measuresthe position of the image input means 303 to thereby acquire positionalinformation (step S32). The directional information acquiring means 302detects the direction of the image input means 303 to thereby acquiredirectional information (step S33). The information transmitting means304 transmits the image data, the positional information, and thedirectional information from the image input means 303, the positionalinformation acquiring means 301, and the directional informationacquiring means 302 via a wireless communication network to thein-fixed-station device 300 b (step S34).

When the in-fixed-station device 300 b executes the characterrecognition processing, which will be described later, and transmitsresult information of the character recognition, the mobile terminaldevice 300 a receives the result information of the characterrecognition via the wireless communication network from thein-fixed-station device 300 b (step S35). And the result information ofthe character recognition is used by the user of the mobile station orthe like, on the side of the mobile terminal device 300 a. For example,the device 300 a displays the received result information of thecharacter recognition on a display module to thereby supply theinformation to the user. Also, for example, the device 300 a may deliverthe received character recognition result information to a sound andvoice output module to thereby supply the information to the user.

When the character recognition result information is received, themobile terminal device 300 a determines whether an indication of theprocessing end is inputted by a user or the like (step S36). If it isdetermined that the indication of termination of the processing isinputted, the device 300 a terminates the information acquisitionprocessing. If it is not determined that the indication of theprocessing end is inputted, the device 300 a returns again to theprocessing of step S31 to repeatedly execute the processing from stepS31 to step S36.

Next, description will be given of operation of the in-fixed-stationdevice 300 b. The information receiving means 305 receives, via awireless communication network from the information transmitting means304 of the mobile terminal device 300 a, the scene image data, thepositional information, and the directional information (step S41). Thecharacter string recognizing means 307 extracts from the map database306, word knowledge possibly included in the scene image according tothe received positional information and the received directionalinformation. Then, the means 307 recognizes characters in the sceneimage using the extracted word knowledge (step S42).

When the character recognition is finished, the in-fixed-station device300 b transmits result information of the character recognition via thewireless communication network to the mobile terminal device 300 a (stepS43).

When the character recognition result information is transmitted, thein-fixed-station device 300 b determines whether an indication of theprocessing end is inputted by a person-in-charge or the like of thefixed station (step S44). If it is determined that the indication of theprocessing end is inputted, the device 300 b terminates the characterrecognition processing. If it is not determined that the indication ofthe processing end is inputted, the device 300 b returns again to theprocessing of step S41 to repeatedly execute the processing from stepS41 to step S44.

As above, according to the embodiment, since it is not necessary for themobile terminal device 300 a to include the map database and thecharacter recognizing means, the configuration of the mobile terminaldevice can be simplified.

Incidentally, in the embodiment, the in-fixed-station device 300 b mayfurther include a document database. In this case, the character stringrecognizing means 307 of the device 300 b may extract documentinformation from the document database using as a retrieval key a wordextracted from the map database 306. And the means 307 may conduct thecharacter string recognition using the document information. Through theoperation, the character recognition accuracy can be increased forcharacter strings not directly related to the positional information.

Embodiment 4

Next, referring to drawings, description will be given of a fourthembodiment of the present invention. FIG. 15 is a block diagram showingan example of the configuration of a mobile communication system inwhich a mobile terminal device and an in-fixed-station device conductcommunication via a wireless transmission path with each other. As shownin FIG. 15, the mobile communication system includes a mobile terminaldevice 400 a installed in a movable mobile station (e.g., a carnavigation terminal device mounted on a vehicle) and an in-fixed-stationdevice 400 b installed in a fixed station. Moreover, the device 400 aincludes positional information acquiring means 401, directionalinformation acquiring means 402, image input means 403, positional anddirectional information transmitting means 404, document informationreceiving means 405, character string recognizing means 406, and a mapdatabase on mobile terminal side 407. Furthermore, the in-fixed-stationdevice 400 b includes positional and directional information receivingmeans 408, document database retrieving means 409, a map database onfixed station side 410, a map database 411, and document informationtransmitting means 412.

Description will be given of a function of the mobile terminal device400 a. The image input means 403 is realized by a mobile video camera orthe like. The means 403 outputs scene image data to the character stringrecognizing means 406.

The positional information acquiring means 401 is realized by a GPSreceiver or the like. The means 401 measures a position of the imageinput means 403 when a scene is shot to thereby obtainlatitude/longitude (positional information). Then, the means 401delivers the obtained positional information to the positional anddirectional information transmitting means 404.

The directional information acquiring means 402 is realized by agyroscope or the like. The means 402 detects a direction (east, west,south, north) of the image input means 403 when the scene is shot tothereby obtain directional information. Then, the means 402 outputs theobtained directional information to the positional and directionalinformation transmitting means 404.

The positional and directional information transmitting means 404 isrealized by a computer (not shown) operating according to a characterrecognition program, a network interface module (not shown) of themobile terminal device 400 a, and the like. In this regard, thecharacter recognition program is stored in a storage device (not shown)of the device 400 a. The positional and directional informationtransmitting means 404 transmits information (positional information anddirectional information) inputted from the positional informationacquiring means 401 and the directional information acquiring means 402via a wireless communication network to the positional and directionalinformation receiving means 408 of the in-fixed-station device 400 b.

The document information receiving means 405 is realized by a computeroperating according to a character recognition program, a networkinterface module of the mobile terminal device 400 a, and the like. Themeans 405 receives document information via a wireless communicationnetwork from the in-fixed-station device 400 b to output the informationto the character string recognizing means 406.

The character string recognizing means 406 is realized by a computeroperating according to a character recognition program. The characterstring recognizing means 204 conducts the character recognitionprocessing using also the document information received from thein-fixed-station device 400 b in addition to the word knowledgeextracted from the map database on mobile terminal side 407.

In this connection, the function and the implementing method of the mapdatabase on mobile terminal side 407 are almost the same as the mapdatabase 203 shown in the second embodiment.

In the embodiment, the storage device of the mobile terminal device 400a has stored various programs to execute the character recognitionprocessing. For example, the storage device of the device 400 a storestherein a character recognition program to cause a computer to executeprocessing for transmitting shooting position information indicating ashooting position of an image and shooting direction informationindicating a shooting direction of an image, via a communication networkto the fixed station device; processing for receiving from the fixedstation device non-corresponding-to-position information which is wordsnot associated with respective positional information indicatingpositions of respective places and is determined by the fixed stationdevice using the shooting position information and the shootingdirection information; and processing for recognizing a character or acharacter string included in the image shot by the mobile terminaldevice, using the non-corresponding-to-position information receivedfrom the fixed station device.

Next, description will be given of a function of the in-fixed-stationdevice (fixed station device) 400 b. The positional and directionalinformation receiving means 408 is implemented by a computer (not shown)operating according to an information extraction program, which will bedescribed later, a network interface module (not shown) of thein-fixed-station device 400 b, and the like. In this regard, theinformation extraction program is stored in a storage device (not shown)of the device 400 b. The positional and directional informationreceiving means 408 receives, via a wireless communication network,information (positional information and directional information)transmitted from the positional and directional information transmittingmeans 404. And the positional and directional information receivingmeans 408 delivers the received information to a document databaseretrieving means 409.

The functions and the implementing methods of the map database on fixedstation side 410 and the document database 411 are respectively almostthe same as for the map database 203 and the document database 205 shownin the second embodiment.

The document database retrieving means 409 is realized by a computer(not shown) or the like operating according to an information extractionprogram. The means 409 makes retrieval through the map database on fixedstation side 410 according to the positional information and thedirectional information received by the positional and directionalinformation receiving means 408, and extracts word knowledge from thedatabase 410. Also, the means 409 makes retrieval through the documentdatabase 411 using the extracted word knowledge as a retrieval key toextract a related document. Then the means 409 outputs the extracteddocument information to the document information transmitting means 412.

In the embodiment, the storage device of the fixed station device 400 bhas stored various programs to conduct the information extractionprocessing. For example, the storage device of the device 400 b storestherein an information extraction program to cause a computer to executeprocessing for receiving from the mobile station device shootingposition information indicating a shooting position of an image shot bythe mobile station device and shooting direction information indicatinga shooting direction of the image; processing for determining, accordingto the shooting position information and the shooting directioninformation that are received from the mobile station device, a range ofan object shot by the mobile station device; processing for extracting,from position corresponding information storage means on fixed stationside storing therein position corresponding information that is wordsassociated with respective positional information indicating positionsof respective places, position corresponding information associated withpositions included in the range; processing for extracting, fromnon-corresponding-to-position information storage means storing thereinnon-corresponding-to-position information that is words not associatedwith positional information, non-corresponding-to-position informationaccording to the position corresponding information extracted from theposition corresponding information storage means on fixed station side;and processing for transmitting the extractednon-corresponding-to-position information via a communication network tothe mobile station device.

Moreover, in the embodiment, the imaging means is realized by the imageinput means 403. The position measuring means is realized by thepositional information acquiring means 401. The direction detectingmeans is realized by the directional information acquiring means 402.Mobile terminal side information transmitting means is realized by thepositional and directional information transmitting means 404. Characterrecognizing means and position corresponding information extractingmeans on mobile terminal side are realized by the character-stringrecognizing means 406. Position corresponding storage means on fixedstation side is realized by the map database on fixed station side 410.Position corresponding information extracting means on fixed stationside and non-corresponding-to-position information extracting means arerealized by the document retrieving means 409.Non-corresponding-to-position information storage means is realized bythe document database 411. Information transmitting means on fixedstation side is realized by the document information transmitting means412. Position corresponding information storage means on mobile stationside is realized by the map database on mobile station side 407.

Next, the operation will be described. FIG. 16 is a flowchart showing anexample of the processing progress of the character recognitionprocessing conducted by the mobile terminal device 400 a. Moreover, FIG.17 is a flowchart showing an example of the processing progress of thedocument information extraction processing executed by thein-fixed-station device 400 b to extract document information.

First, description will be given of operation of the mobile terminaldevice 400 a. The positional information acquiring means 401 measuresthe position of the image input means 403 to thereby acquire positionalinformation (step S52). The directional information acquiring means 402detects the direction of the image input means 403 to thereby acquiredirectional information (step S53). The positional and directionalinformation transmitting means 404 transmits the positional informationand the directional information from the positional informationacquiring means 401 and the directional information acquiring means 402via a wireless communication network to the in-fixed-station device 400b (step S54).

When the in-fixed-station device 400 b executes the document informationextraction processing, which will be described later, and transmitsdocument information, the document information receiving means 405receives the document information via the wireless communication networkfrom the in-fixed-station device 400 b (step S55).

In this regard, after having transmitted the positional information andthe directional information in step S54, the mobile terminal device 400a enters a wait state and stays therein without executing processing ofthe next step S56 until document information is received from thein-fixed-station device 400 b. In this situation, it may also bepossible that the mobile terminal device 400 a executes, while awaitingprocessing, in advance processing executable without using the documentinformation from the in-fixed-station device 400 b. For example, evenbefore the document information is received, the device 400 a mayconduct character recognition by assuming a range of one character inthe background image in various ways.

When the document information is received, the character stringrecognizing means 406 recognizes a character string in the image datafrom the image input means 403 by use of the word knowledge extractedfrom the map database on mobile terminal side 407 based on thepositional information and the directional information and the documentinformation received from the in-fixed-station device 400 b (step S56).In this connection, the method of extracting word knowledge such as abuilding name and a store name from the map database on mobile terminalside 407 according to the positional information and the directionalinformation is similar to that shown in the first embodiment.

When the character recognition is finished, the mobile station device400 a determines whether an indication of the processing end is inputtedby a user or the like (step S57). If it is determined that theindication of the processing end is inputted, the device 400 aterminates the character recognition processing. If it is not determinedthat the indication of the processing end is inputted, the device 400 areturns again to the processing of step S51 to repeatedly execute theprocessing from step S51 to step S57.

The character recognition result information is used by the user or thelike of a mobile station on the side of the mobile terminal device 400a. For example, device 400 a may display the result information on adisplay module (not shown) to thereby supply the information to theuser. Moreover, for example, the device 400 a may output the resultinformation to a voice and sound output module (not shown) or the liketo thereby supply the information to the user.

Next, description will be given of operation of the in-fixed-stationdevice 400 b. The positional and directional information receiving means408 receives, via the wireless communication network, the positionalinformation and the directional information sent from the positional anddirectional information transmitting means 404 of the mobile terminaldevice 400 a in step S54 (step S61). Then the means 408 outputs thereceived positional and directional information to the document databaseretrieving means 409.

The document database retrieving means 409 refers to the map database onfixed station side 410 according to the received positional anddirectional information to extract word knowledge from the database 410.The means 409 makes retrieval through the document database 411 usingthe extracted word knowledge as a retrieval key to extract relateddocument information from the document database 411 (step S62). Then,the means 409 outputs the extracted document information to the documentinformation transmitting means 412.

The document information transmitting means 412 transmits the documentinformation via a wireless communication network to the mobile terminaldevice 400 a (step S63). After the document is transmitted, thein-fixed-station device 400 b determines whether an indication of theprocessing end is inputted by a person-in-charge or the like of thefixed station (step S64). If it is determined that the indication of theprocessing end is inputted, the device 400 b terminates the characterrecognition processing. If it is not determined that the indication ofthe processing end is inputted, the device 400 b returns again to theprocessing of step S61 to repeatedly execute the processing from stepS61 to step S64.

As above, according to the embodiment, the in-fixed-station device 400 bincludes the document database 411 to execute the retrieval processingof the document information on the side of the device 400 b. Thedocument database retrieval time considerably varies according to thesize of the document database. Also, the document database retrievalprocessing takes a long processing time in comparison with otherprocessing. According to the embodiment, since the document databaseretrieval processing is conducted in the fixed station, the processingload of the mobile terminal device 400 a is mitigated, and in comparisonwith the second embodiment, the computation performance required for themobile terminal device 400 a is equal to or less than that of the secondembodiment. Moreover, since the document database is controlled on thefixed station side in a unified manner, the maintenance is facilitatedand maintainability can be improved.

Incidentally, when the in-fixed-station device 400 b sends informationto the mobile terminal device 400 a, there may also be considered aconfiguration in which the word knowledge extracted from the mapdatabase on fixed station side 410 is also transmitted together with thecharacter information. In the embodiment, the transmission is conductedlimitatively for the document information, and hence the data volumetransmitted from the in-fixed-station device 400 b to the mobileterminal device 400 a can be reduced.

Furthermore, in the embodiment, description has been given of a case inwhich the in-fixed-station device 400 b transmits only documentinformation. However, device 400 b may also transmit, in addition to thedocument information, the word knowledge extracted from the map databaseon fixed station side 410 to the mobile terminal device 400 a. In thissituation, it is also possible for the device 400 a not to include themap database on mobile terminal side 407 and conducts the characterrecognition using only the information received from thein-fixed-station device 400 b. Through the operation, the processingload of the mobile terminal device 400 a can be further mitigated incomparison with the case where only the document information istransmitted, and hence high computation performance is not required forthe device 400 a.

INDUSTRIAL APPLICABILITY

The character recognition device in accordance with the presentinvention is applicable to a system including terminal devices such as acar navigation terminal device mounted on a vehicle and a portabletelephone. For example, when the device is applied to a system includinga car navigation device or the like, a message that a signboardindicating a destination has been detected can be notified to thedriver. Also, when a route to the destination is guided to the driver,road guide information can be supplied by indication of a road guidesign or the like. Additionally, peripheral information matching thetaste of a driver or the like can be provided to the driver while a caris moving. For example, using a character recognition result from thecharacter recognition device, it is possible to provide the driver withinformation such as “a signboard of a target noodle shop is seen”, “turnto the left at an intersection where the next guide sign is standing(toward Ginza)”, and “exhibition of your favorite painter is being heldin a department store ahead on the right”.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of the configuration of acharacter recognition device in accordance with the present invention.

FIG. 2 is a flowchart showing an example of the character recognitionprocessing executed by the character recognition device.

FIG. 3 is an explanatory diagram showing an example of the particularrange determined by the character recognition device.

FIG. 4 is an explanatory diagram showing an example of thetransportation network information regarding roads.

FIG. 5 is an explanatory diagram showing an example of the place nameinformation.

FIG. 6 is an explanatory diagram showing an example of the buildinginformation.

FIG. 7 is an explanatory diagram showing an example of the storeinformation.

FIG. 8 is an explanatory diagram in which the current position, thecurrent direction, and the shooting range of the character recognitiondevice at a certain point of time are displayed in an overlapping way ona map.

FIG. 9 is a block diagram showing another example of the configurationof the character recognition device.

FIG. 10 is a flowchart showing another example of the characterrecognition processing executed by the character recognition device.

FIG. 11 is an explanatory diagram showing an example of a hanging screenof a department store shot by a vehicle mounted camera of a vehicle.

FIG. 12 is a block diagram showing an example of the configuration of amobile communication system in which a mobile terminal device and anin-fixed-office device communicate via a wireless transmission path witheach other.

FIG. 13 is a flowchart showing an example of the information acquisitionprocessing for the mobile terminal device 300 a to acquire positionalinformation, directional information, and scene image data.

FIG. 14 is a flowchart showing an example of the character recognitionprocessing which the in-fixed-station device 300 a executes.

FIG. 15 is a block diagram showing another example of the configurationof a mobile communication system in which a mobile terminal device andan in-fixed-office device communicate via a wireless transmission pathwith each other.

FIG. 16 is a flowchart showing an example of the character recognitionprocessing executed by the mobile terminal device 400 a.

FIG. 17 is a flowchart showing an example of the document informationextraction processing executed by the in-fixed-station device 400 b toextract document information.

DESCRIPTION OF REFERENCE NUMERALS

-   101 Positional information acquiring means-   102 Directional information acquiring means-   103 Map database-   104 Character string recognizing means-   106 Image input means

The invention claimed is:
 1. A mobile communication system in which amobile terminal device and a fixed station device communicate via awireless transmission path with each other, characterized in that: themobile terminal device comprises: imaging means for shooting an image;position measuring means for measuring a shooting position of an imageto obtain shooting position information indicating the shootingposition; direction detecting means for detecting a shooting directionof an image to obtain shooting direction information indicating theshooting direction; and information transmitting means on mobileterminal side for transmitting the shooting position information, theshooting direction information, and an image shot by the imaging meansvia a communication network to the fixed station device; and the fixedstation device comprises: position corresponding information storagemeans for storing therein position corresponding information that iswords associated with respective positional information indicatingpositions of respective places; position corresponding informationextracting means for determining, based on the shooting positioninformation and the shooting direction information that are receivedfrom the information transmitting means on mobile terminal side, a rangeof a shooting object of the imaging means, and extracting from theposition corresponding information storage means position correspondinginformation associated with positions included in the range; characterrecognizing means for recognizing, using the position correspondinginformation extracted by the position corresponding informationextracting means, a character or a character string included in theimage that are received from the information transmitting means onmobile terminal side; and information transmitting means on fixedstation side for transmitting information of a character or a characterstring recognized by the character recognizing means via a communicationnetwork to the mobile terminal device.
 2. The mobile communicationsystem in accordance with claim 1, wherein the fixed station devicecomprises: non-corresponding-to-position information storage means forstoring therein non-corresponding-to-position information that is wordsnot associated with positional information; andnon-corresponding-to-position information extracting means forextracting non-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means, wherein the characterrecognizing means recognizes a character or a character string includedin the image using the position corresponding information extracted bythe position corresponding information extracting means and thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means.
 3. Themobile communication system in accordance with claim 1, wherein themobile terminal device comprises: information receiving means on mobileterminal side for receiving information of a character or a characterstring recognized by the character recognizing means via a communicationnetwork from the information transmitting means on fixed station side;and output means for outputting therefrom information of the characteror the character string.
 4. A mobile communication system in which amobile terminal device and a fixed station device communicate via awireless transmission path with each other, characterized in that: themobile terminal device comprises: imaging means for shooting an image;position measuring means for measuring a shooting position of an imageto obtain shooting position information indicating the shootingposition; direction detecting means for detecting a shooting directionof an image to obtain shooting direction information indicating theshooting direction; information transmitting means on mobile terminalside for transmitting the shooting position information and the shootingdirection information via a communication network to the fixed stationdevice; and character recognizing means for recognizing a character or acharacter string included in the image shot by the imaging means; andthe fixed station device comprises: position corresponding informationstorage means on fixed station side for storing therein positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places;position corresponding information extracting means on fixed stationside for determining, based on the shooting position information and theshooting direction information that are received from the informationtransmitting means on mobile terminal side, a range of a shooting objectof the imaging means, and extracting from the position correspondinginformation storage means on fixed station side position correspondinginformation associated with positions included in the range;non-corresponding-to-position information storage means for storingtherein non-corresponding-to-position information that is words notassociated with positional information; non-corresponding-to-positioninformation extracting means for extractingnon-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means on fixed station side; andinformation transmitting means on fixed station side for transmittingthe non-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means via acommunication network to the mobile terminal device, wherein thecharacter recognizing means recognizes a character or a character stringincluded in the image using the non-corresponding-to-positioninformation received from the information transmitting means on fixedstation side.
 5. The mobile communication system in accordance withclaim 4, wherein: the mobile terminal device comprises: positioncorresponding information storage means on mobile terminal side forstoring position corresponding information that is words associated withrespective positional information indicating positions of respectiveplaces; and position corresponding information extracting means onmobile terminal side for determining, based on the shooting positioninformation and the shooting direction information, a range of ashooting object of the imaging means, and extracting from the positioncorresponding information storage means on mobile terminal side positioncorresponding information associated with positions included in therange, wherein the character recognizing means recognizes a character ora character string included in the image using thenon-corresponding-to-position information received from the informationtransmitting means on fixed station side and the position correspondinginformation extracted by the position corresponding informationextracting means on mobile terminal side.
 6. The mobile communicationsystem in accordance with claim 4, wherein: the information transmittingmeans on fixed station side transmits the non-corresponding-to-positioninformation extracted by the non-corresponding-to-position informationextracting means and the position corresponding information extracted bythe position corresponding information extracting means on fixed stationside via a communication network to the mobile terminal device; and thecharacter recognizing means recognizes a character or a character stringincluded in the image using the non-corresponding-to-positioninformation and the position corresponding information that are receivedfrom the information transmitting means on fixed station side.
 7. Themobile communication system in accordance with claim 4, wherein: thenon-corresponding-to-position information storage means storesnon-corresponding-to-position information that is a set of words notassociated with positional information and that includes at least oneword equal to position corresponding information; and thenon-corresponding-to-position information extracting means extractsnon-corresponding-to-position information from thenon-corresponding-to-position information storage means using as a keythe position corresponding information extracted by the positioncorresponding information extracting means on fixed station side.
 8. Amobile terminal device for communicating information with a fixedstation device that executes character recognition processing torecognize a character or a character string included in an image,comprising: imaging means for shooting an image; position measuringmeans for measuring a shooting position of an image to obtain shootingposition information indicating the shooting position; directiondetecting means for detecting a shooting direction of an image to obtainshooting direction information indicating the shooting direction;information transmitting means on mobile terminal side for transmittingthe shooting position information, the shooting direction information,and an image imaged by the imaging means via a communication network tothe fixed station device; and information receiving means on mobileterminal side for receiving from the fixed station device information ofa character or a character string obtained by the fixed station devicethrough the character recognition processing for the image using theshooting position information and the shooting direction information. 9.A fixed station device for communicating information with a mobileterminal device, comprising: information receiving means on fixedstation side for receiving from the mobile terminal device an image shotby the mobile terminal device, the shooting position informationindicating a shooting position of the image, and the shooting directioninformation indicating a shooting direction of the image; positioncorresponding information storage means for storing positioncorresponding information that is words associated with respectivepositional information indicating positions of respective places;position corresponding information extracting means for determining,based on the shooting position information and the shooting directioninformation that are received from the mobile terminal device, a rangeof an object shot by the mobile terminal device, and extracting from theposition corresponding information storage means position correspondinginformation associated with positions included in the range; characterrecognizing means for recognizing a character or a character stringincluded in the image received from the mobile terminal device, usingthe position corresponding information extracted by the positioncorresponding information extracting means; and information transmittingmeans on fixed station side for transmitting information of a characteror a character string recognized by the character recognizing means viaa communication network to the mobile terminal device.
 10. The fixedstation device in accordance with claim 9, comprising:non-corresponding-to-position information storage means for storingnon-corresponding-to-position information that is words not associatedwith positional information; and non-corresponding-to-positioninformation extracting means for extractingnon-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means, wherein the characterrecognizing means recognizes a character or a character string includedin the image using the position corresponding information extracted bythe position corresponding information extracting means and thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means.
 11. A mobileterminal device for communicating information with a fixed stationdevice and executing character recognition processing to recognize acharacter or a character string included in an image, comprising:imaging means for shooting an image; position measuring means formeasuring a shooting position of an image to obtain shooting positioninformation indicating the shooting position; direction detecting meansfor detecting a shooting direction of an image to obtain shootingdirection information indicating the shooting direction; informationtransmitting means on mobile terminal side for transmitting the shootingposition information and the shooting direction information via acommunication network to the fixed station device; information receivingmeans on mobile terminal side for receiving from the fixed stationdevice non-corresponding-to-position information that is words notassociated with respective positional information indicating positionsof respective places and that is determined by the fixed station deviceusing the shooting position information and the shooting directioninformation; and character recognizing means for recognizing a characteror a character string included in the image imaged by the imaging means,using the non-corresponding-to-position information received by theinformation receiving means on mobile terminal side.
 12. The mobileterminal device in accordance with claim 11, comprising: positioncorresponding information storage means on mobile terminal side forstoring position corresponding information that is words associated withrespective positional information indicating positions of respectiveplaces; and position corresponding information extracting means onmobile terminal side for determining, based on the shooting positioninformation and the shooting direction information, a range of ashooting object of the imaging means, and extracting from the positioncorresponding information storage means on mobile terminal side positioncorresponding information associated with positions included in therange, wherein the character recognizing means recognizes a character ora character string included in the image using thenon-corresponding-to-position information received by the informationreceiving means on mobile terminal side and the position correspondinginformation extracted by the position corresponding informationextracting means on mobile terminal side.
 13. A fixed station device forcommunicating information with a mobile terminal device that executescharacter recognition processing to recognize a character or a characterstring included in an image, comprising: information receiving means onfixed station side for receiving from the mobile terminal device theshooting position information of an image that is shot by the mobileterminal device and that indicates a shooting position of the image andthe shooting direction information indicating a shooting direction ofthe image; position corresponding information storage means on fixedstation side for storing therein position corresponding information thatis words associated with respective positional information indicatingpositions of respective places; position corresponding informationextracting means on fixed station side for determining, based on theshooting position information and the shooting direction informationthat are received by the information receiving means on fixed stationside, a range of an object shot by the mobile terminal device, andextracting from the position corresponding information storage means onfixed station side position corresponding information associated withpositions included in the range; non-corresponding-to-positioninformation storage means for storing non-corresponding-to-positioninformation that is words not associated with positional information;non-corresponding-to-position information extracting means forextracting non-corresponding-to-position information from thenon-corresponding-to-position information storage means according to theposition corresponding information extracted by the positioncorresponding information extracting means on fixed station side; andinformation transmitting means on fixed station side for transmittingthe non-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means via acommunication network to the mobile terminal device.
 14. The fixedstation device in accordance with claim 13, wherein the informationtransmitting means on fixed station side transmits thenon-corresponding-to-position information extracted by thenon-corresponding-to-position information extracting means and theposition corresponding information extracted by the positioncorresponding information extracting means on fixed station side.
 15. Acharacter recognition method of recognizing a character or a characterstring included in an image, for use in a mobile communication system inwhich a mobile terminal device and a fixed station device communicatevia a wireless transmission path with each other, comprising: the mobileterminal device shooting an image; the mobile terminal device measuringa shooting position of an image to obtain shooting position informationindicating the shooting position; the mobile terminal device detecting ashooting direction of an image to obtain shooting direction informationindicating the shooting direction; the mobile terminal devicetransmitting the shooting position information, the shooting directioninformation, and an image imaged by the imaging means via acommunication network to the fixed station device; the fixed stationdevice determining, based on the shooting position information and theshooting direction information that are received from the mobileterminal device, a range of a shooting object of the mobile terminaldevice; the fixed station device extracting, from position correspondinginformation storage means storing position corresponding informationthat is words associated with respective positional informationindicating positions of respective places, position correspondinginformation associated with positions included in the range; the fixedstation device recognizing, using the extracted position correspondinginformation, a character or a character string included in the imagereceived from the mobile terminal device; and the fixed station devicetransmits information of a character or a character string via acommunication network to the mobile terminal device.
 16. The characterrecognition method in accordance with claim 15, further comprising: themobile terminal device receiving information of a character or acharacter string recognized by the fixed station device, via acommunication network from the fixed station device; and the mobileterminal device outputting information of the character or the characterstring.
 17. A character recognition method of recognizing a character ora character string included in an image, for use in a mobilecommunication system in which a mobile terminal device and a fixedstation device communicate via a wireless transmission path with eachother, comprising: the mobile terminal device shooting an image; themobile terminal device measuring a shooting position of an image toobtain shooting position information indicating the shooting position;the mobile terminal device detecting a shooting direction of an image toobtain shooting direction information indicating the shooting direction;the mobile terminal device transmitting the shooting positioninformation and the shooting direction information via a communicationnetwork to the fixed station device; the fixed station devicedetermining, based on the shooting position information and the shootingdirection information that are received from the mobile terminal device,a range of a shooting object of the mobile terminal device; the fixedstation device extracting, from position corresponding informationstorage means on fixed station side storing position correspondinginformation that is words associated with respective positionalinformation indicating positions of respective places, positioncorresponding information associated with positions included in therange; the fixed station device extracting, fromnon-corresponding-to-position information storage means storingnon-corresponding-to-position information that is words not associatedwith positional information, non-corresponding-to-position informationaccording to the position corresponding information extracted from theposition corresponding information storage means on fixed station side;the fixed station device transmitting the extractednon-corresponding-to-position information via a wireless transmissionnetwork to the mobile terminal device; and the mobile station devicerecognizing a character or a character string included in the image shotby the mobile terminal device, using the non-corresponding-to-positioninformation received from the fixed station device.
 18. A non-transitorycomputer readable medium storing a character recognition program, whichcauses a computer to execute: determining, based on shooting positioninformation indicating a shooting position of an image and shootingdirection information indicating a shooting direction of an image, arange of an object of imaging; extracting, from position correspondinginformation storage means storing position corresponding informationthat is words associated with respective positional informationindicating positions of respective places, position correspondinginformation associated with positions included in the range; andrecognizing, using the extracted position corresponding information, acharacter or a character string included in the image thus shot.
 19. Thenon-transitory computer readable medium of claim 18, wherein thecharacter recognition program cases the computer to further execute:extracting, from non-corresponding-to-position information storage meansstoring non-corresponding-to-position information that is words notassociated with positional information, non-corresponding-to-positioninformation according to the position corresponding informationextracted from the position corresponding information storage means; andrecognizing a character or a character string included in the imageusing the position corresponding information extracted from the positioncorresponding information storage means and thenon-corresponding-to-position information extracted from thenon-corresponding-to-position information storage means.
 20. Aninformation communication program in a mobile terminal devicecommunicating information with a fixed station device that recognizes acharacter or a character string included in an image, causing a computerto execute: transmitting shooting position information indicating ashooting position of an image, shooting direction information indicatinga shooting direction of an image, and an image via a wirelesstransmission network to the fixed station device; and receiving from thefixed station device information of a character or a character stringobtained by the fixed station device through the character recognitionprocessing for the image using the shooting position information and theshooting direction information.
 21. A character recognition program in afixed station device for communicating information with a mobileterminal device, causing a computer to execute: receiving from themobile terminal device the image shot by the mobile terminal device, theshooting position information indicating a shooting position of theimage, and the shooting direction information indicating a shootingdirection of the image; determining, based on the shooting positioninformation and the shooting direction information that are receivedfrom the mobile terminal device, a range of an object shot by the mobileterminal device; extracting, from position corresponding informationstorage means storing therein position corresponding information that iswords associated with respective positional information indicatingpositions of respective places, position corresponding informationassociated with positions included in the range; recognizing a characteror a character string included in the image received from the mobileterminal device, using the extracted position corresponding information;and transmitting information of a character or a character string via acommunication network to the mobile terminal device.
 22. A characterrecognition program in a mobile terminal device for communicatinginformation with a fixed station device and recognizing a character or acharacter string included in an image, causing a computer to execute:transmitting shooting position information indicating a shootingposition of an image and shooting direction information indicating ashooting direction of an image via a communication network to the fixedstation device; receiving from the fixed station devicenon-corresponding-to-position information that is words not associatedwith respective positional information indicating positions ofrespective places and which is determined by the fixed station deviceusing the shooting position information and the shooting directioninformation; and recognizing a character or a character string includedin the image shot by the mobile terminal device, using thenon-corresponding-to-position information received from the fixedstation device.
 23. An information extraction program in a fixed stationdevice for communicating information with a mobile terminal device whichrecognizes a character or a character string included in an image,causing a computer to execute: receiving from the mobile terminal deviceshooting position information indicating a shooting position of an imageshot by the mobile terminal device and shooting direction informationindicating a shooting direction of the image; determining, based on theshooting position information and the shooting direction informationthat are received from the mobile terminal device, a range of an objectshot by the mobile terminal device; extracting, from positioncorresponding information storage means on fixed station side storingposition corresponding information that is words associated withrespective positional information indicating positions of respectiveplaces, position corresponding information associated with positionsincluded in the range; extracting, from non-corresponding-to-positioninformation storage means storing non-corresponding-to-positioninformation that is words not associated with positional information,non-corresponding-to-position information according to the positioncorresponding information extracted from the position correspondinginformation storage means on fixed station side; and transmitting theextracted non-corresponding-to-position information via a communicationnetwork to the mobile terminal device.